The present invention generally relates to a technique for compressing images (image signals), and in particular, to hierarchical image coding/decoding devices and hierarchical image coding/decoding methods by which lossless decoding of coded images is realized.
As a reversible image coding method, the present inventor has recently proposed a technique in Japanese Patent Application Laid-Open No.HEI10-108186 (published Apr. 24, 1998): xe2x80x9cTransform Coding Method for Digital Signals enabling Reversible Transformation.xe2x80x9d The technique is a reversible image coding method based on discrete cosine transform (DCT), and enables reversible coding by improving an ordinary coding method which employs discrete cosine transform. A reversible image coding method can also be constructed by applying the above technique to an MPEG-2 (Moving Picture Experts Group-Phase 2) coding method.
By combining the above reversible image coding method and the MPEG-2 SNR scalability together, a reversible hierarchical coding method can be constructed, as we mentioned in R. Oami and M. Ohta xe2x80x9cA Study on Hierarchical Coding with Lossless DCT,xe2x80x9d The Proceedings of the 12th Picture Coding Symposium of Japan, pages 107-108 (October 1997). By the reversible hierarchical coding method, two layer bit-streams, a base layer bit-stream and an enhancement layer bit-stream, are generated. The base layer bit-stream can be decoded with an ordinary MPEG-2 decoder. The enhancement layer bit-stream is a bit-stream for complementing the base layer bit-stream. A lossless decoded image which is exactly the same as the original image can be obtained by decoding the base layer bit-stream and the enhancement layer bit-stream using inverse transformation of the reversible discrete cosine transform described in the Japanese Patent Application Laid-Open No.HEI10-108186 (published Apr. 24, 1998).
FIG. 1 is a circuit diagram showing an example of a hierarchical image coding device which is realized by employing the MPEG-2 SNR scalability and the reversible discrete cosine transform proposed by the present inventors. Referring to FIG. 1, the hierarchical image coding device comprises a quantization control section 3, a quantizer 4, an inverse quantizer (de-quantizer) 5, a motion estimation/compensation circuit (ME/MC) 50, a reversible discrete cosine transform circuit (RDCT) (lossless discrete cosine transform circuit (LDCT)) 51, code conversion sections 14 and 52 subtracters 1 and 53, and multiplexers 54 and 55.
The motion estimation/compensation circuit 50 executes motion compensation to an input video signal (the original image signal), and thereby generates and outputs a motion compensated prediction image signal. At the same time, the motion estimation/compensation circuit 50 also outputs macro block information. The subtracter 1 subtracts the motion compensated prediction image signal outputted by the motion estimation/compensation circuit 50 from the input video signal (the original image signal), and thereby outputs a prediction error image signal. The reversible discrete cosine transform circuit 51 executes the reversible discrete cosine transform to the prediction error image signal outputted by the subtracter 1, and thereby obtains transform coefficients of the reversible discrete cosine transform.
The quantization control section 3 determines and outputs a quantization scale on every macro block, based on the prediction error image signal outputted by the subtracter 1 and code quantity information of the base layer bit-stream outputted by the multiplexer 54. The quantizer 4 quantizes the transform coefficients outputted by the reversible discrete cosine transform circuit 51 based on a quantization matrix, the quantization scale determined by the quantization control section 3, and the macro block information outputted by the motion estimation/compensation circuit 50, and thereby outputs the quantized values of the transform coefficients to the code conversion section 14.
The code conversion section 14 encodes the quantized values outputted by the quantizer 4 into a variable length code based on the macro block information outputted by the motion estimation/compensation circuit 50 and a predetermined coding table, and thereby outputs the variable length code of the quantized values to the multiplexer 54. The multiplexer 54 multiplexes the variable length code outputted by the code conversion section 14, the quantization scale outputted by the quantization control section 3, the macro block information outputted by the motion estimation/compensation circuit 50, the quantization matrix and other additional information, and thereby generates and outputs the base layer bit-stream. The multiplexer 54 also outputs the code quantity information of the base layer bit-stream to the quantization control section 3.
The inverse quantizer 5 inversely quantizes (de-quantizes) the quantized values outputted by the quantizer 4, using the quantization matrix and the quantization scale and based on the macro block information outputted by the motion estimation/compensation circuit 50, and outputs the inversely quantized values to the subtracter 53. The subtracter 53 subtracts the inversely quantized values outputted by the inverse quantizer 5 from the transform coefficients outputted by the reversible discrete cosine transform circuit 51, and thereby obtains coding residual values (coding residual signal). The code conversion section 52 encodes the coding residual values outputted by the subtracter 53 into a variable length code based on a predetermined coding table, and thereby outputs the variable length code of the coding residual values to the multiplexer 55. The multiplexer 55 multiplexes the variable length code outputted by the code conversion section 52 and other additional information, and thereby generates and outputs the enhancement layer bit-stream.
Incidentally, while motion compensation is generally executed using a locally decoded image signal in ordinary MPEG-2 image coding devices, the image coding device of FIG. 1 does not have such a component. That is because local decoding is unnecessary since a decoded image (an image signal after decoding) becomes exactly the same as the original image (the original image signal before coding) in the case where the reversible discrete cosine transform is employed instead of the ordinary discrete cosine transform.
FIG. 2 is a circuit diagram showing an example of a hierarchical image decoding device for decoding the video signal (image signal) which has been coded by the hierarchical image coding device of FIG. 1. Referring to FIG. 2, the hierarchical image decoding device comprises demultiplexers 72 and 73, code conversion sections 30 and 70, an inverse quantizer (de-quantizer) 32, adders 34 and 36, a reversible discrete cosine inverse transform circuit (RDCIT) 71, and a motion compensation circuit (MC) 37.
The demultiplexer 72 demultiplexes the base layer bit-stream into the variable length code of the quantized values, the quantization scale, the macro block information, the quantization matrix and the additional information. The code conversion section 30 decodes the variable length code of the quantized values using the predetermined coding table, and thereby outputs the quantized values to the inverse quantizer 32. The. inverse quantizer 32 executes inverse quantization (de-quantization), which is specified by the quantization scale, the macro block information and the quantization matrix outputted by the demultiplexer 72, to the quantized values outputted by the code conversion section 30, and thereby obtains the inversely quantized values.
The demultiplexer 73 demultiplexes the enhancement layer bit-stream into the variable length code of the coding residual values and the additional information. The code conversion section 70 decodes the variable length code of the coding residual values using the predetermined coding table, and thereby obtains the coding residual values. The adder 34 adds the inversely quantized values outputted by the inverse quantizer 32 and the coding residual values outputted by the code conversion section 70, and outputs the added values to the reversible discrete cosine inverse transform circuit 71 as the reversible discrete cosine transform coefficients. The reversible discrete cosine inverse transform circuit 71 executes inverse transformation of the reversible discrete cosine transform to the reversible discrete cosine transform coefficients outputted by the adder 34, and thereby obtains and outputs the prediction error image signal. The adder 36 adds the prediction error image signal outputted by the reversible discrete cosine inverse transform circuit 71 and a motion compensated prediction image signal outputted by the motion compensation circuit 37, and thereby obtains and outputs a decoded image signal. The motion compensation circuit 37 executes motion compensation to the decoded image signal outputted by the adder 36 based on the macro block information outputted by the demultiplexer 72, and thereby obtains and outputs the motion compensated prediction image signal.
Meanwhile, a method for encoding the enhancement layer of MPEG-2 SNR scalability into an arithmetic code and thereby improving coding efficiency has been disclosed in D. Wilson et al., xe2x80x9cEfficient Coding Methods for Enhancement DCT Coefficients in SNR Scalable Codersxe2x80x9d, Proceedings of PCS 97, pages 407-410 (September 1997). The method employs arithmetic coding for generating the enhancement layer bit-stream, instead of employing the variable length coding of MPEG-2.
In addition, an image coding method disclosed in Japanese Patent Application Laid-Open No.HEI4-72870 executes orthogonal transform to an image and executes arithmetic coding to transform coefficients which are obtained by the orthogonal transform. In the method, orthogonal transform coding and quantization are executed to each block of an image, and the arithmetic coding is executed to the quantized values. For the arithmetic coding, occurrence probability density functions representing the probability of occurrence of each possible value of each of the quantized values (symbols) are preliminarily prepared. In the arithmetic coding, the occurrence probability density function of a quantized value is switched or changed adaptively depending on local characteristics of the block.
However, the devices and methods described above involves the following problems or drawbacks.
In the hierarchical image coding device of FIG. 1 which employs the reversible coding method of the Japanese Patent Application Laid-Open No.HEI10-108186 and the MPEG-2 SNR scalability, actual code quantity of the enhancement layer bit-stream becomes considerably large in comparison with entropy. The large code quantity of the enhancement layer bit-stream occurs by the following reasons.
First, in high bit rate coding that is required in the reversible coding, occurrence probability distribution of each quantization residual value in the enhancement layer becomes widely different from occurrence probability distribution which has been supposed when the variable length coding of MPEG-2 was designed.
Second, the coding method employed in the variable length coding of MPEG-2, in which combination of a 0-run length (the number of successive zeros) and a level (the absolute value of the next non-zero value) is encoded, becomes inefficient at such a high bit rate. Incidentally, entropy can generally be reduced by repeating entropy coding of coding residual values with respect to each frequency index (i,j) and thereby completing entropy coding with respect to all the frequency indexes (i,j), in comparison with the case where entropy coding is executed to the combination of the 0-run length and the level. The frequency index (i,j) is an index indicating the position of a transform coefficient in a transform coefficient matrix which is obtained by reversible transform such as the reversible discrete cosine transform executed by the reversible discrete cosine transform circuit 51 of the hierarchical image coding device of FIG. 1.
Meanwhile, it is possible to improve coding efficiency by executing arithmetic coding in the enhancement layer as disclosed in the document of D. Wilson et al. However, the method employs a fixed quantization step (quantization scale) of the base layer, and thus the method can not handle cases where the quantization scale of the base layer is changed adaptively block by block.
In the method disclosed in Japanese Patent Application Laid-Open No.HEI4-72870 in which the occurrence probability density function of the quantized value is switched depending on local characteristics of the block, adequate occurrence probability density functions have to be preliminarily prepared with respect to each of the quantized values, and the occurrence probability density function has to be appropriately selected depending on characteristics of the quantization in order to realize improvement of the coding efficiency. Further, even if we tried to apply the method to the enhancement layer, the occurrence probability density function of the coding residual values widely varies depending on the quantization executed in the base layer. Therefore, it is difficult to execute adaptive switching of the occurrence probability density function in the enhancement layer and improve the coding efficiency.
It is therefore the primary object of the present invention to provide a hierarchical image coding device and a hierarchical image decoding device, by which reversible image coding with improved coding efficiency can be realized.
Another object of the present invention is to provide a hierarchical image coding method and a hierarchical image decoding method, by which reversible image coding with improved coding efficiency can be realized.
In accordance with a 1st aspect of the present invention, there is provided a hierarchical image coding device for encoding an original image signal hierarchically, comprising a first coding means, a coding residual value occurrence probability calculation means, a second coding means and a multiplexing means. The first coding means encodes the original image signal and outputs a base layer bit-stream, quantization information and coding residual values. The coding residual value occurrence probability calculation means calculates occurrence probability of each possible value of each of the coding residual values, using the quantization information. The second coding means encodes the coding residual values based on the coding residual value occurrence probabilities calculated by the coding residual value occurrence probability calculation means and thereby outputs coding residual value code. The multiplexing means multiplexes the coding residual value code and the quantization information, and thereby generates and outputs an enhancement layer bit-stream.
In accordance with a 2nd aspect of the present invention, in the 1st aspect, the coding residual values are coding residual values which occur when the first coding means executes quantization to transform coefficients which are obtained by executing reversible transform to the original image signal. The quantization information which is outputted by the first coding means and multiplexed by the multiplexing means into the enhancement layer bit-stream includes quantization scale frequency distribution and a quantization matrix. The coding residual value occurrence probability calculation means calculates the coding residual value occurrence probabilities, based on the quantization scale frequency distribution, the quantization matrix, and frequency distribution of each transform coefficient which has preliminarily been prepared. The second coding means executes variable length coding to the coding residual values based on the coding residual value occurrence probabilities, in which variable length coding of coding residual values concerning each frequency index (i,j) is repeated and thereby variable length coding concerning all the frequency indexes (i,j) is completed.
In accordance with a 3rd aspect of the present invention, in the 2nd aspect, the variable length coding executed by the second coding means is arithmetic coding.
In accordance with a 4th aspect of the present invention, in the 1st aspect, the coding residual values are coding residual values which occur when the first coding means executes quantization to transform coefficients which are obtained by executing reversible transform to the original image signal. The quantization information which is outputted by the first coding means includes quantization scale frequency distribution and a quantization matrix, and the quantization information which is multiplexed by the multiplexing means into the enhancement layer bit-stream includes the quantization scale frequency distribution. The coding residual value occurrence probability calculation means calculates the coding residual value occurrence probabilities, based on the quantization scale frequency distribution, the quantization matrix, and frequency distribution of each transform coefficient which has preliminarily been prepared. The second coding means executes variable length coding to the coding residual values based on the coding residual value occurrence probabilities, in which variable length coding of coding residual values concerning each frequency index (i,j) is repeated and thereby variable length coding concerning all the frequency indexes (i,j) is completed.
In accordance with a 5th aspect of the present invention, in the 4th aspect, the variable length coding executed by the second coding means is arithmetic coding.
In accordance with a 6th aspect of the present invention, in the 1st aspect, the first coding means includes a transform coefficient collection means. The transform coefficient collection means collects transform coefficients which are obtained by executing reversible transform to the original image signal, and thereby obtains and outputs frequency distribution of each of the transform coefficients. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution stored therein using the transform coefficient frequency distribution outputted by the transform coefficient collection means, and calculates the coding residual value occurrence probabilities using the updated transform coefficient frequency distribution.
In accordance with a 7th aspect of the present invention, in the 1st aspect, the first coding means includes a first image signal generation means, a second image signal generation means, a transform means, a quantization scale setting means, a quantization means, a variable length coding means, a base layer multiplexing means, an inverse quantization means, a coding residual value calculation means and a quantization scale collection means. The first image signal generation means executes motion estimation and motion compensation to the original image signal and thereby generates a motion compensated prediction image signal. The first image signal generation means further outputs macro block information. The second image signal generation means subtracts the motion compensated prediction image signal from the original image signal and thereby generates a prediction error image signal. The transform means executes reversible transform to the prediction error image signal and thereby obtains transform coefficients. The quantization scale setting means sets a quantization scale for each macro block based on the prediction error image signal. The quantization means executes quantization to the transform coefficients and thereby outputs quantized values, based on the quantization scale, a quantization matrix and the macro block information. The variable length coding means executes variable length coding to the quantized values based on the macro block information and a predetermined coding table, and thereby outputs a variable length code of the quantized values. The base layer multiplexing means multiplexes the variable length code of the quantized values, the quantization matrix, the quantization scale and the macro block information, and thereby generates and outputs the base layer bit-stream. The inverse quantization means executes inverse quantization to the quantized values based on the quantization scale, the quantization matrix and the macro block information, and thereby outputs inversely quantized values. The coding residual value calculation means subtracts the inversely quantized values from the transform coefficients, and thereby obtains the coding residual values. The quantization scale collection means collects the quantization scales at predetermined coding periods, and thereby obtains and outputs quantization scale frequency distribution. The coding residual value occurrence probability calculation means calculates the coding residual value occurrence probabilities based on the quantization scale frequency distribution outputted by the quantization scale collection means and the quantization matrix.
In accordance with an 8th aspect of the present invention, in the 7th aspect, the base layer multiplexing means outputs code quantity information of the base layer bit-stream as well as outputting the base layer bit-stream, and the quantization scale setting means sets the quantization scale for each of the macro blocks, based on the prediction error image signal and the code quantity information.
In accordance with a 9th aspect of the present invention, in the 7th aspect, the quantization scale collection means executes the collection of the quantization scales, with respect to each different block type separately, based on the macro block information, and thereby obtains and outputs quantization scale frequency distribution with respect to each of the different block types. The coding residual value occurrence probability calculation means calculates coding residual value occurrence probabilities with respect to each of the different block types separately, using the quantization scale frequency distribution with respect to each block type outputted by the quantization scale collection means. The second coding means executes the coding of the coding residual values, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 10th aspect of the present invention, in the 9th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with an 11th aspect of the present invention, in the 7th aspect, the first coding means further includes a transform coefficient collection means. The transform coefficient collection means collects the transform coefficients obtained by the transform means and thereby obtains and outputs frequency distribution of each of the transform coefficients. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution stored therein using the transform coefficient frequency distribution outputted by the transform coefficient collection means, and calculates the coding residual value occurrence probabilities using the updated transform coefficient frequency distribution.
In accordance with a 12th aspect of the present invention, in the 11th aspect, the transform coefficient collection means executes the collection of the transform coefficients, with respect to each different block type separately, based on the macro block information, and thereby obtains and outputs transform coefficient frequency distribution with respect to each of the different block types. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution with respect to each of the different block types stored therein using the transform coefficient frequency distribution with respect to each of the different block types outputted by the transform coefficient collection means, and calculates coding residual value occurrence probabilities with respect to each of the different block types separately, using the. updated transform coefficient frequency distribution with respect to each of the different block types. The second coding means executes the coding of the coding residual values, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 13th aspect of the present invention, in the 12th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 14th aspect of the present invention, in the 1st aspect, the first coding means includes a first image signal generation means, a second image signal generation means, a first transform means, a quantization scale setting means, a quantization means, a variable length coding means, a base layer multiplexing means, an inverse quantization means, a third image signal generation means, a fourth image signal generation means, a fifth image signal generation means, a sixth image signal generation means, a second transform means, a coding residual value calculation means and a quantization scale collection means. The first image signal generation means executes motion estimation between the original image signal and a locally decoded image signal, executes motion compensation to the locally decoded image signal based on the result of the motion estimation, and thereby generates a first motion compensated prediction image signal. The first image signal generation means further outputs macro block information. The second image signal generation means subtracts the first motion compensated prediction image signal from the original image signal and thereby generates a first prediction error image signal. The first transform means executes reversible transform to the first prediction error image signal and thereby obtains first transform coefficients. The quantization scale setting means sets a quantization scale for each macro block based on the first prediction error image signal. The quantization means executes quantization to the first transform coefficients and thereby outputs quantized values, based on the quantization scale, a quantization matrix and the macro block information. The variable length coding means executes variable length coding to the quantized values based on the macro block information and a predetermined coding table, and thereby outputs a variable length code of the quantized values. The base layer multiplexing means multiplexes the variable length code of the quantized values, the quantization matrix, the quantization scale and the macro block information, and thereby generates and outputs the base layer bit-stream. The inverse quantization means executes inverse quantization to the quantized values based on the quantization scale, the quantization matrix and the macro block information, and thereby outputs inversely quantized values. The third image signal generation means executes inverse transformation of the reversible transform to the inversely quantized values, and thereby generates a locally decoded prediction error image signal. The fourth image signal generation means adds the locally decoded prediction error image signal to the first motion compensated prediction image signal, and thereby generates the locally decoded image signal. The fifth image signal generation means executes motion compensation to the original image signal based on the macro block information, and thereby generates a second motion compensated prediction image signal. The sixth image signal generation means subtracts the second motion compensated prediction image signal from the original image signal, and thereby generates a second prediction error image signal. The second transform means executes the reversible transform to the second prediction error image signal and thereby obtains second transform coefficients. The coding residual value calculation means subtracts the inversely quantized values from the second transform coefficients, and thereby obtains the coding residual values. The quantization scale collection means collects the quantization scales at predetermined coding periods, and thereby obtains and outputs quantization scale frequency distribution. The coding residual value occurrence probability calculation means calculates the coding residual value occurrence probabilities based on the quantization scale frequency distribution outputted by the quantization scale collection means and the quantization matrix.
In accordance with a 15th aspect of the present invention, in the 14th aspect, the base layer multiplexing means outputs code quantity information of the base layer bit-stream as well as outputting the base layer bit-stream, and the quantization scale setting means sets the quantization scale for each of the macro blocks, based on the first prediction error image signal and the code quantity information.
In accordance with a 16th aspect of the present invention, in the 14th aspect, the quantization scale collection means executes the collection of the quantization scales, with respect to each different block type separately, based on the macro block information, and thereby obtains and outputs quantization scale frequency distribution with respect to each of the different block types. The coding residual value occurrence probability calculation means calculates coding residual value occurrence probabilities with respect to each of the different block types separately, using the quantization scale frequency distribution with respect to each block type outputted by the quantization scale collection means. The second coding means executes the coding of the coding residual values, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 17th aspect of the present invention, in the 16th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with an 18th aspect of the present invention, in the 14th aspect, the first coding means further includes a transform coefficient collection means. The transform coefficient collection means collects the second transform coefficients obtained by the second transform means and thereby obtains and outputs frequency distribution of each of the second transform coefficients. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution stored therein using the second transform coefficient frequency distribution outputted by the transform coefficient collection means, and calculates the coding residual value occurrence probabilities using the updated transform coefficient frequency distribution.
In accordance with a 19th aspect of the present invention, in the 18th aspect, the transform coefficient collection means executes the collection of the second transform coefficients, with respect to each different block type separately, based on the macro block information, and thereby obtains and outputs transform coefficient frequency distribution with respect to each of the different block types. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution with respect to each of the different block types stored therein using the transform coefficient frequency distribution with respect to each of the different block types outputted by the transform coefficient collection means, and calculates coding residual value occurrence probabilities with respect to each of the different block types separately, using the updated transform coefficient frequency distribution with respect to each of the different block types. The second coding means executes the coding of the coding residual values, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 20th aspect of the present invention, in the 19th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 21st aspect of the present invention, there is provided a hierarchical image decoding device for decoding the base layer bit-stream and the enhancement layer bit-stream which have been encoded by the hierarchical image coding device of the above aspects. The hierarchical image decoding device comprises a demultiplexing means, a coding residual value occurrence probability calculation means, a first decoding means and a second decoding means. The demultiplexing means demultiplexes the coding residual value code and the quantization information from the enhancement layer bit-stream. The coding residual value occurrence probability calculation means calculates occurrence probability of each possible value of each of the coding residual values, using the quantization information. The first decoding means decodes the coding residual value code based on the coding residual value occurrence probabilities calculated by the coding residual value occurrence probability calculation means, and thereby outputs the coding residual values. The second decoding means decodes the base layer bit-stream, adds the coding residual values to a signal obtained by decoding the base layer bit-stream, and thereby regenerates the original image signal.
In accordance with a 22nd aspect of the present invention, in the 21st aspect, the coding residual value occurrence probability calculation means calculates the coding residual value occurrence probabilities, based on quantization scale frequency distribution included in the quantization information demultiplexed from the enhancement layer bit-stream, a quantization matrix, and frequency distribution of each transform coefficient which has preliminarily been prepared. The first decoding means executes variable length decoding to the coding residual value code based on the coding residual value occurrence probabilities, in which variable length decoding of coding residual value code concerning each frequency index (i,j) is repeated and thereby variable length decoding concerning all the frequency indexes (i,j) is completed.
In accordance with a 23rd aspect of the present invention, in the 22nd aspect, the variable length decoding executed by the first decoding means is arithmetic decoding.
In accordance with a 24th aspect of the present invention, in the 21st aspect, the second decoding means includes a transform coefficient collection means. The transform coefficient collection means collects transform coefficients which are obtained by executing the decoding of the base layer bit-stream and adding the coding residual values to the decoding result, and thereby obtains and outputs frequency distribution of each of the transform coefficients. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution stored therein using the transform coefficient frequency distribution outputted by the transform coefficient collection means, and calculates the coding residual value occurrence probabilities using the updated transform coefficient frequency distribution.
In accordance with a 25th aspect of the present invention, in the 21st aspect, the second decoding means includes a base layer demultiplexing means, a variable length decoding means, an inverse quantization means, a transform coefficient calculation means, a first image signal regeneration means, a second image signal regeneration means and a third image signal regeneration means. The base layer demultiplexing means demultiplexes the variable length code of the quantized values, the quantization matrix, the quantization scale and the macro block information from the base layer bit-stream. The variable length decoding means executes variable length decoding to the variable length code of the quantized values based on the macro block information and a predetermined coding table, and thereby outputs the quantized values. The inverse quantization means executes inverse quantization to the quantized values based on the quantization matrix, the quantization scale and the macro block information, and thereby outputs the inversely quantized values. The transform coefficient calculation means adds the coding residual values outputted by the first decoding means and the inversely quantized values outputted by the inverse quantization means, and thereby outputs the transform coefficients. The first image signal regeneration means executes inverse transformation of the reversible transform that has been executed in the hierarchical image coding device to the transform coefficients, and thereby regenerates the prediction error image signal. The second image signal regeneration means adds the prediction error image signal to a motion compensated prediction image signal, and thereby generates a decoded image signal. The third image signal regeneration means executes motion compensation to the decoded image signal based on the macro block information, and thereby generates the motion compensated prediction image signal.
In accordance with a 26th aspect of the present invention, in the 25th aspect, the quantization information demultiplexed from the enhancement layer bit-stream by the demultiplexing means includes quantization scale frequency distribution with respect to each different block type. The coding residual value occurrence probability calculation means calculates coding residual value occurrence probabilities with respect to each of the different block types separately, using the quantization scale frequency distribution with respect to each block type. The first decoding means executes the decoding of the coding residual value code, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual value code to be decoded.
In accordance with a 27th aspect of the present invention, in the 26th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 28th aspect of the present invention, in the 25th aspect, the second decoding means further includes a transform coefficient collection means. The transform coefficient collection means collects transform coefficients which are obtained by executing the decoding of the base layer bit-stream and adding the coding residual values to the decoding result, and thereby obtains and outputs frequency distribution of each of the transform coefficients. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution stored therein using the transform coefficient frequency distribution outputted by the transform coefficient collection means, and calculates the coding residual value occurrence probabilities using the updated transform coefficient frequency distribution.
In accordance with a 29th aspect of the present invention, in the 28th aspect, the transform coefficient collection means executes the collection of the transform coefficients, with respect to each different block type separately, based on the macro block information, and thereby obtains and outputs transform coefficient frequency distribution with respect to each of the different block types. The coding residual value occurrence probability calculation means successively updates transform coefficient frequency distribution with respect to each of the different block types stored therein using the transform coefficient frequency distribution with respect to each of the different block types outputted by the transform coefficient collection means, and calculates coding residual value occurrence probabilities with respect to each of the different block types separately, using the updated transform coefficient frequency distribution with respect to each of the different block types. The first decoding means executes the decoding of the coding residual value code, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual value code to be decoded.
In accordance with a 30th aspect of the present invention, in the 29th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 31st aspect of the present invention, there is provided a hierarchical image coding method for encoding an original image signal hierarchically, comprising a first coding step, a coding residual value occurrence probability calculation step, a second coding step and a multiplexing step. In the first coding step, the original image signal is encoded and thereby a base layer bit-stream, quantization information and coding residual values are generated. In the coding residual value occurrence probability calculation step, occurrence probability of each possible value of each of the coding residual values is calculated using the quantization information. In the second coding step, the coding residual values are encoded based on the coding residual value occurrence probabilities calculated in the coding residual value occurrence probability calculation step and thereby coding residual value code is generated. In the multiplexing step, the coding residual value code and the quantization information are multiplexed and thereby an enhancement layer bit-stream is generated.
In accordance with a 32nd aspect of the present invention, in the 31st aspect, the coding residual values are coding residual values which occur in the first coding step when quantization is executed to transform coefficients which are obtained by executing reversible transform to the original image signal. The quantization information which is generated in the first coding step and multiplexed into the enhancement layer bit-stream in the multiplexing step includes quantization scale frequency distribution and a quantization matrix. In the coding residual value occurrence probability calculation step, the coding residual value occurrence probabilities are calculated based on the quantization scale frequency distribution, the quantization matrix, and frequency distribution of each transform coefficient which has preliminarily been prepared. In the second coding step, variable length coding is executed to the coding residual values based on the coding residual value occurrence probabilities, in which variable length coding of coding residual values concerning each frequency index (i,j) is repeated and thereby variable length coding concerning all the frequency indexes (i,j) is completed.
In accordance with a 33rd aspect of the present invention, in the 32nd aspect, the variable length coding executed in the second coding step is arithmetic coding.
In accordance with a 34th aspect of the present invention, in the 31st aspect, the coding residual values are coding residual values which occur in the first coding step when quantization is executed to transform coefficients which are obtained by executing reversible transform to the original image signal. The quantization information which is generated in the first coding step includes quantization scale frequency distribution and a quantization matrix, and the quantization information which is multiplexed into the enhancement layer bit-stream in the multiplexing step includes the quantization scale frequency distribution. In the coding residual value occurrence probability calculation step, the coding residual value occurrence probabilities are calculated based on the quantization scale frequency distribution, the quantization matrix, and frequency distribution of each transform coefficient which has preliminarily been prepared. In the second coding step, variable length coding is executed to the coding residual values based on the coding residual value occurrence probabilities, in which variable length coding of coding residual values concerning each frequency index (i,j) is repeated and thereby variable length coding concerning all the frequency indexes (i,j) is completed.
In accordance with a 35th aspect of the present invention, in the 34th aspect, the variable length coding executed in the second coding step is arithmetic coding.
In accordance with a 36th aspect of the present invention, in the 31st aspect, the first coding step includes a transform coefficient collection step. In the transform coefficient collection step, transform coefficients which are obtained by executing reversible transform to the original image signal are collected, and thereby frequency distribution of each of the transform coefficients is obtained. In the coding residual value occurrence probability calculation step, transform coefficient frequency distribution successively updated by the transform coefficient frequency distribution obtained by the transform coefficient collection step is used for calculating the coding residual value occurrence probabilities.
In accordance with a 37th aspect of the present invention, in the 31st aspect, the first coding step includes a first image signal generation step, a second image signal generation step, a transform step, a quantization scale setting step, a quantization step, a variable length coding step, a base layer multiplexing step, an inverse quantization step, a coding residual value calculation step and a quantization scale collection step. In the first image signal generation step, motion estimation and motion compensation are executed to the original image signal and thereby a motion compensated prediction image signal is generated and macro block information is obtained. In the second image signal generation step, the motion compensated prediction image signal is subtracted from the original image signal and thereby a prediction error image signal is generated. In the transform step, reversible transform is executed to the prediction error image signal and thereby transform coefficients are obtained. In the quantization scale setting step, a quantization scale is set for each macro block based on the prediction error image signal. In the quantization step, quantization is executed to the transform coefficients based on the quantization scale, a quantization matrix and the macro block information, and thereby quantized values are obtained. In the variable length coding step, variable length coding is executed to the quantized values based on the macro block information and a predetermined coding table, and thereby a variable length code of the quantized values is obtained. In the base layer multiplexing step, the variable length code of the quantized values, the quantization matrix, the quantization scale and the macro block information are multiplexed, and thereby the base layer bit-stream is generated. In the inverse quantization step, inverse quantization is executed to the quantized values based on the quantization scale, the quantization matrix and the macro block information, and thereby inversely quantized values are obtained. In the coding residual value calculation step, the inversely quantized values are subtracted from the transform coefficients, and thereby the coding residual values are obtained. In the quantization scale collection step, the quantization scales are collected at predetermined coding periods, and thereby quantization scale frequency distribution is obtained, In the coding residual value occurrence probability calculation step, the coding residual value occurrence probabilities are calculated based on the quantization scale frequency distribution obtained in the quantization scale collection step and the quantization matrix.
In accordance with a 38th aspect of the present invention, in the 37th aspect, code quantity information of the base layer bit-stream is obtained as well as generating the base layer bit-stream in the base layer multiplexing step. In the quantization scale setting step, the quantization scale is set for each of the macro blocks, based on the prediction error image signal and the code quantity information.
In accordance with a 39th aspect of the present invention, in the 37th aspect, the collection of the quantization scales is executed with respect to each different block type separately, based on the macro block information, and thereby quantization scale frequency distribution with respect to each of the different block types are obtained in the quantization scale collection step. In the coding residual value occurrence probability calculation step, coding residual value occurrence probabilities with respect to each of the different block types are calculated separately, using the quantization scale frequency distribution with respect to each block type obtained in the quantization scale collection step. In the second coding step, the coding of the coding residual values is executed, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 40th aspect of the present invention, in the 39th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 41st aspect of the present invention, in the 37th aspect, the first coding step further includes a transform coefficient collection step. In the transform coefficient collection step, the transform coefficients obtained in the transform step are collected and thereby frequency distribution of each of the transform coefficients is obtained. In the coding residual value occurrence probability calculation step, transform coefficient frequency distribution successively updated by the transform coefficient frequency distribution obtained by the transform coefficient collection step is used for calculating the coding residual value occurrence probabilities.
In accordance with a 42nd aspect of the present invention, in the 41st aspect, the collection of the transform coefficients is executed with respect to each different block type separately, based on the macro block information, and thereby transform coefficient frequency distribution with respect to each of the different block types is obtained in the transform coefficient collection step. In the coding residual value occurrence probability calculation step, coding residual value occurrence probabilities with respect to each of the different block types are calculated separately, using transform coefficient frequency distribution with respect to each of the different block types successively updated by the transform coefficient frequency distribution with respect to each of the different block types obtained by the transform coefficient collection step. In the second coding step, the coding of the coding residual values is executed, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 43rd aspect of the present invention, in the 42nd aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 44th aspect of the present invention, in the 31st aspect, the first coding step includes a first image signal generation step, a second image signal generation step, a first transform step, a quantization scale setting step, a quantization step, a variable length coding step, a base layer multiplexing step, an inverse quantization step, a third image signal generation step, a fourth image signal generation step, a fifth image signal generation step, a sixth image signal generation step, a second transform step, a coding residual value calculation step and a quantization scale collection step. In the a first image signal generation step, motion estimation is executed between the original image signal and a locally decoded image signal, motion compensation is executed to the locally decoded image signal based on the result of the motion estimation, and thereby a first motion compensated prediction image signal is generated, and macro block information is obtained. In the second image signal generation step, the first motion compensated prediction image signal is subtracted from the original image signal and thereby a first prediction error image signal is generated. In the first transform step, reversible transform is executed to the first prediction error image signal and thereby first transform coefficients are obtained. In the quantization scale setting step, a quantization scale is set for each macro block based on the first prediction error image signal. In the quantization step, quantization is executed to the first transform coefficients based on the quantization scale, a quantization matrix and the macro block information, and thereby quantized values are obtained. In the variable length coding step, variable length coding is executed to the quantized values based on the macro block information and a predetermined coding table, and thereby a variable length code of the quantized values is generated. In the base layer multiplexing step, the variable length code of the quantized values, the quantization matrix, the quantization scale and the macro block information are multiplexed, and thereby the base layer bit-stream is generated. In the inverse quantization step, inverse quantization is executed to the quantized values based on the quantization scale, the quantization matrix and the macro block information, and thereby inversely quantized values are obtained. In the third image signal generation step, inverse transformation of the reversible transform is executed to the inversely quantized values, and thereby a locally decoded prediction error image signal is generated. In the fourth image signal generation step, the locally decoded prediction error image signal is added to the first motion compensated prediction image signal, and thereby the locally decoded image signal is generated. In the fifth image signal generation step, motion compensation is executed to the original image signal based on the macro block information, and thereby a second motion compensated prediction image signal is generated. In the sixth image signal generation step, the second motion compensated prediction image signal is subtracted from the original image signal, and thereby a second prediction error image signal is generated. In the second transform step, the reversible transform is executed to the second prediction error image signal and thereby second transform coefficients are obtained. In the coding residual value calculation step, the inversely quantized values are subtracted from the second transform coefficients, and thereby the coding residual values are obtained. In the quantization scale collection step, the quantization scales are collected at predetermined coding periods, and thereby quantization scale frequency distribution is obtained. In the coding residual value occurrence probability calculation step, the coding residual value occurrence probabilities are calculated based on the quantization scale frequency distribution obtained in the quantization scale collection step and the quantization matrix.
In accordance with a 45th aspect of the present invention, in the 44th aspect, code quantity information of the base layer bit-stream is obtained as well as generating the base layer bit-stream in the base layer multiplexing step. In the quantization scale setting step, the quantization scale is set for each of the macro blocks, based on the first prediction error image signal and the code quantity information.
In accordance with a 46th aspect of the present invention, in the 44th aspect, the collection of the quantization scales is executed with respect to each different block type separately, based on the macro block information, and thereby quantization scale frequency distribution with respect to each of the different block types is obtained in the quantization scale collection step. In the coding residual value occurrence probability calculation step, coding residual value occurrence probabilities with respect to each of the different block types are calculated separately, using the quantization scale frequency distribution with respect to each block type obtained in the quantization scale collection step. In the second coding step, the coding of the coding residual values is executed, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 47th aspect of the present invention, in the 46th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 48th aspect of the present invention, in the 44th aspect, the first coding step further includes a transform coefficient collection step. In the transform coefficient collection step, the second transform coefficients obtained in the second transform step are collected and thereby frequency distribution of each of the second transform coefficients is obtained. In the coding residual value occurrence probability calculation step, transform coefficient frequency distribution successively updated by the second transform coefficient frequency distribution obtained by the transform coefficient collection step is used for calculating the coding residual value occurrence probabilities.
In accordance with a 49th aspect of the present invention, in the 48th aspect, the collection of the second transform coefficients is executed with respect to each different block type separately, based on the macro block information, and thereby transform coefficient frequency distribution with respect to each of the different block types is obtained in the transform coefficient collection step. In the coding residual value occurrence probability calculation step, coding residual value occurrence probabilities with respect to each of the different block types are calculated separately, using transform coefficient frequency distribution with respect to each of the different block types successively updated by the transform coefficient frequency distribution with respect to each of the different block types obtained by the transform coefficient collection step. In the second coding step, the coding of the coding residual values is executed, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual values to be coded.
In accordance with a 50th aspect of the present invention, in the 49th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 51st aspect of the present invention, there is provided a hierarchical image decoding method for decoding the base layer bit-stream and the enhancement layer bit-stream which have been encoded by the hierarchical image coding method of the above aspects, comprising a demultiplexing step, a coding residual value occurrence probability calculation step, a first decoding step and a second decoding step. In the demultiplexing step, the coding residual value code and the quantization information are demultiplexed from the enhancement layer bit-stream. In the coding residual value occurrence probability calculation step, occurrence probability of each possible value of each of the coding residual values is calculated using the quantization information. In the first decoding step, the coding residual value code is decoded based on the coding residual value occurrence probabilities calculated in the coding residual value occurrence probability calculation step, and thereby the coding residual values are obtained. In the second decoding step, the base layer bit-stream is decoded, the coding residual values are added to a signal obtained by decoding the base layer bit-stream, and thereby the original image signal is regenerated.
In accordance with a 52nd aspect of the present invention, in the coding residual value occurrence probability calculation step of the 51st aspect, the coding residual value occurrence probabilities are calculated based on quantization scale frequency distribution included in the quantization information demultiplexed from the enhancement layer bit-stream, a quantization matrix, and frequency distribution of each transform coefficient which has preliminarily been prepared. In the first decoding step, variable length decoding is executed to the coding residual value code based on the coding residual value occurrence probabilities, in which variable length decoding of coding residual value code concerning each frequency index (i,j) is repeated and thereby variable length decoding concerning all the frequency indexes (i,j) is completed.
In accordance with a 53rd aspect of the present invention, in the 52nd aspect, the variable length decoding executed in the first decoding step is arithmetic decoding.
In accordance with a 54th aspect of the present invention, in the 51st aspect, the second decoding step includes a transform coefficient collection step. In the transform coefficient collection step, transform coefficients which are obtained by executing the decoding of the base layer bit-stream and adding the coding residual values to the decoding result is collected and thereby frequency distribution of each of the transform coefficients is obtained. In the coding residual value occurrence probability calculation step, transform coefficient frequency distribution successively updated by the transform coefficient frequency distribution obtained by the transform coefficient collection step is used for calculating the coding residual value occurrence probabilities.
In accordance with a 55th aspect of the present invention, in the 51st aspect, the second decoding step includes a base layer demultiplexing step, a variable length decoding step, an inverse quantization step, a transform coefficient calculation step, a first image signal regeneration step, a second image signal regeneration step and a third image signal regeneration step. In the base layer demultiplexing step, the variable length code of the quantized values, the quantization matrix, the quantization scale and the macro block information are demultiplexed from the base layer bit-stream. In the variable length decoding step, variable length decoding is executed to the variable length code of the quantized values based on the macro block information and a predetermined coding table, and thereby the quantized values are obtained. In the inverse quantization step, inverse quantization is executed to the quantized values based on the quantization matrix, the quantization scale and the macro block information, and thereby the inversely quantized values are obtained. In the transform coefficient calculation step, the coding residual values obtained in the first decoding step and the inversely quantized values obtained in the inverse quantization step are added, and thereby the transform coefficients are obtained. In the first image signal regeneration step, inverse transformation of the reversible transform that has been executed in the hierarchical image coding method is executed to the transform coefficients, and thereby the prediction error image signal is regenerated. In the second image signal regeneration step, the prediction error image signal is added to a motion compensated prediction image signal, and thereby a decoded image signal is generated. In the third image signal regeneration step, motion compensation is executed to the decoded image signal based on the macro block information, and thereby the motion compensated prediction image signal is generated.
In accordance with a 56th aspect of the present invention, in the 55th aspect, the quantization information demultiplexed from the enhancement layer bit-stream in the demultiplexing step includes quantization scale frequency distribution with respect to each different block type. In the coding residual value occurrence probability calculation step, coding residual value occurrence probabilities with respect to each of the different block types are calculated separately, using the quantization scale frequency distribution with respect to each block type. In the first decoding step, the decoding of the coding residual value code is executed, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual value code to be decoded.
In accordance with a 57th aspect of the present invention, in the 56th aspect, the different block types are intra blocks and non-intra blocks.
In accordance with a 58th aspect of the present invention, in the 55th aspect, the second decoding step further includes a transform coefficient collection step. In the transform coefficient collection step, transform coefficients which are obtained by executing the decoding of the base layer bit-stream and adding the coding residual values to the decoding result are collected and thereby frequency distribution of each of the transform coefficients is obtained. In the coding residual value occurrence probability calculation step, transform coefficient frequency distribution successively updated by the transform coefficient frequency distribution obtained by the transform coefficient collection step is used for calculating the coding residual value occurrence probabilities.
In accordance with a 59th aspect of the present invention, in the 58th aspect, the collection of the transform coefficients is executed with respect to each different block type separately, based on the macro block information, and thereby transform coefficient frequency distribution with respect to each of the different block types is obtained in the transform coefficient collection step. In the coding residual value occurrence probability calculation step, coding residual value occurrence probabilities with respect to each of the different block types are calculated separately, using transform coefficient frequency distribution with respect to each of the different block types successively updated by the transform coefficient frequency distribution with respect to each of the different block types obtained by the transform coefficient collection step. In the first decoding step, the decoding of the coding residual value code is executed, referring to the macro block information and using coding residual value occurrence probabilities with respect to a block type corresponding to the coding residual value code to be decoded.
In accordance with a 60th aspect of the present invention, in the 59th aspect, the different block types are intra blocks and non-intra blocks.